Matrix factorization

Intuition

Say, we have $n \times m$ matrix, where each column is a variable and row is a person, document, or other entities. Then we can think of a decomposition that breaks this measurement matrix into two matrices. The first one ($n \times r$) captures entity -> hidden factors ($r$) then the second one ($r \times m$) explains what are the hidden factors or how the hidden factors manifest in measurements.

$$\boldsymbol{M}=\boldsymbol{A B}^{T} \quad \text { with } \quad \boldsymbol{M} \in \mathbb{R}^{n \times m}, \boldsymbol{A} \in \mathbb{R}^{n \times r}, \boldsymbol{B}^{T} \in \mathbb{R}^{r \times m}$$ - Matrix factorizations are the pinnacle results of linear algebra. This is how to make sense of them.

Methods

• Singular value decomposition
• Non negative matrix factorization

Generalization

• Tensor decomposition